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Abstract. This paper develops a decision algorithm for weak bisimula- 
tion on Markov Automata (MA). For that purpose, different notions of 
vanishing state (a concept known from the area of Generalised Stochas- 
tic Petri Nets) are defined. Vanishing states are shown to be essential 
for relating the concepts of (state-based) naive weak bisimulation and 
(distribution-based) weak bisimulation. The bisimulation algorithm pre- 
sented here follows the partition-refinement scheme and has exponential 
time complexity. 



1 Introduction 

Markov Automata (MA) are a powerful formalism for modelling systems with 
nondeterminism, probability and continuous time. Unfortunately up to now no 
algorithm for weak bisimulation on MA has been published. The contribution of 
this paper is a transfer of the notion of vanishing states, as known in the area 
of Generalised Stochastic Petri Net, to the MA setting. This allows us to state 
a decision algorithm for weak MA bisimulation. 

Since MA have been defined in 2010 [3] it remained an open problem how 
to decide weak MA bisimilarity. As naive weak bisimulation on MA corresponds 
to weak probabilistic bisimulation on Probabilistic Automata (PA), naive weak 
MA bisimulation is known to be decidable since 2002 pQ. There, an exponential 
time algorithm was presented. In 2012 a polynomial time algorithm has been 
presented for deciding naive weak MA bisimulation [5]. Our algorithm has ex- 
ponential time complexity (the result of [5] does not seem to be applicable for 
the weak case). 

The paper is organised as follows: In Sec. [5] we present the necessary prelim- 
inaries and recall a mapping from MA to PA from [3J. Sec. [3J recalls some facts 
on weak bisimulation. In Sec. 0] we define different notions of vanishing states 
and use them to relate weak bisimulation and naive weak bisimulation. Finally, 
Sec. [5] describes our partition refinement algorithm that heavily relies on Sec. [4j 



2 Preliminaries 



First we define the notion of discrete subdistribution and related terms and 
notations: 



Definition 1 ((Sub-)distributions). A mapping : S — > [0, 1] is called (dis- 
crete) subdistribution, ifj^sesl 1 ^) — ^ s usua ^ we write fi(S') for ^2 s£S , fJ-(s). 
The support of fx is defined as Supp(fi) := {s G S\li(s) > 0}. The empty subdis- 
tribution fi$ is defined by Supp(fi^) = 0. The size o//i is defined as := ^(S). 
A subdistribution /i is called distribution if = 1. The sets Dist(S) and 
Subdist(S) denote distributions and subdistributions defined over the set S . Let 
A s G Dist(S) denote the Dirac distribution on s, i.e. A s (s) — 1. For two subdis- 
tributions [i, \i' the sum fj," := /U0// is defined as fi"(s) := /i(s) + /i'(s) fas Zong 
fl s I A*" I — !• ^ s l° n 9 o.s c • \n\ < 1, we denote by c[i the subdistribution defined 
by (c/i)(s) := c- fi(s). For a subdistribution \i and a state s G Supp(^i) we define 
H- s by 



The definition of Markov Automata we use is the one from |3I4) . 

Definition 2 (Markov Automata |3j). A Markov automaton MA is a tuple 
(S, Act, — -»-, -»**-, so), where 

— S is a nonempty finite set of states 

— Act is a set of actions containing the internal action t 

— — -»-C S x Actx Dist(S) a set of action-labelled probabilistic transitions (PT) 

— ->**-C S x M>o x S a set of Markovian timed transitions (MT) and 

— Sq G S the initial state 

A state in an MA is called stable if it has no emanating t transitions, otherwise 
it is called unstable. A stable state s will be denoted by si- 
Remark 1. In order to make our decision algorithm feasible we assume in the fol- 
lowing that, in contrast to the original definition from [314] . all sets in Definition 
[2] are finite. 

For simplicity we define probabilistic automata (PA) in terms of MA. 

Definition 3. A probabilistic automaton (PA) is a MA M — (S, Act, — >, 0, so). 

Note that the PA we define are also PA in the sense of [TJ, but not vice versa 
([7j also allows for different actions within one distribution). We do not need the 
more general definition of Segala in this paper. 

For the mapping from MA to PA introduced in [3] we need to define the 
probability distribution on successor states. Remark llGl in the appendix explains 
why this is needed. 

Definition 4 (modified version of Definition 3 in [3j). Let M = (S, Act, — 
sq) be a MA. Define 




for t =/= s 
for t = s 



rate(s, s') := A 

(s,A,s') £-»♦»- 



and rate(s) '■= ^2 s , eS rate(s, s') which is called the exit rate of state s. The 
corresponding probability distributions are defined in the following way: 

Ps:= {-^ ! ^# forrate(s)^0 
I A s otherwise 

We would like to stress that this definition in the original setting from 
is problematic, as for infinite sets *-►► the exit rate may not converge. 



2.1 A mapping from MA to PA 

The remarkable idea of [3J is to define bisimulations on MA using a mapping 
from MA to PA. The basic ingredient is a set of special actions, denoted by 
X(.), that cover timed behaviour. In the setting of 3 4 countable action sets 
are mapped to uncountable action sets by definition, as for every real number a 
new action name is introduced. In order to keep the action set finite, we redefine 
Act x in the context of a fixed MA: 

Definition 5. Let M = (S, Act, —»*-,-»**-, so) be a MA. Assume^r 6 R>o x( r ) £ 
Act and definelZT := {rate(s)\s 6 S} (which is finite). Then we define Act*^^ — 
{x{r)\r S TIT} and Act* := Act U Act* [nr ^ . 

There is a mapping from MA to PA (adapted from [3J) where we use Defini- 
tion m 

Definition 6. Let M = (S,Act, — *+-,-+*+■, so) be a MA. Define the transitions 
— > as follows: For s G S define 

a J ct £ Act and s jj, 

s fiij y s ^ a _ x ( mte ( s )) e Act^ nr '> and (j, = P s 

Then the mapping VA : MA — > PA is defined by M h-> (S*, Act x , — >, 0, so)- 

Note that every timed transition is part of a special x(') action. So the set of 
actions is increased by the mapping PA{-), but no timed transition remains in 
the image. For more details on the procedure we refer to [3J. Note further that 
this is not always well defined. If you have infinitely many Markovian transitions 
emanating from one state, the exit rate does not need to converge. As a result 
you cannot define the corresponding probability distribution correctly. 

Example 1. Two basic examples are given in Fig.[T] Mi (Fig. Ha]) has a r loop, so 
no timed transition (i.e. x(')) exists after the transformation, i.e. VA(Mi) = M\. 
In the example M^ (Fig. Ilb[) u is a stable state and therefore the transformation 
leads to a x transition with exit rate (Fig. [Tc|) . 

Remark 2. The mapping PA(-) is not surjective, as a Markovian race condition 
is always converted to a deterministic x transition. That means for example the 
PA in Fig.[Te]is not in VA(MA). The mapping is also not injective, as M3 ^ M\ 
in Fig. [[J but VA(Mi) = VA{M 3 ) = M x , 




(a) Mi = VA{Mi) (b) M 2 (c) VA(M 2 ) (d) M 3 




(e) PA which is not in VA(MA) 



Fig. 1: MA to PA transformations 
2.2 Weak transitions 

In this section we recall definitions from |3I4) , complete them and correct minor 
typos. 

Definition 7 (Sequences of integers). Let (Z + )* the set of all possibly infi- 
nite sequences of positive integers. For a, a 1 G (Z + )* we write a < a' if there 
exists a (possibly empty) <f> G (Z + )* such that, by usual concatenation, o~(j> = a' . 
If the equality holds for nonempty <j), we also write a < a' 

Definition 8 (Labelled trees, cf. Section II in |3j, Section 2 in [4j). Let 

A C (Z + )* and L be an arbitrary set. A function 

t-.a^l 

is called (infinite) L-labelled tree, if the following conditions hold: 

1. (a 1 e A and a E (Z+)* and cr < a' ) =>• a 6 A 

2. (ai £ A and i > lj=> a(i - 1) £ A 

3. £ G A (empty sequence) 

The elements of a £ A are called nodes of T ' ■ An element a e A is called leaf 
of T if there is no a' £ A such that a < a 1 . The symbol e is called the root 
of T ■ The set of leaves of T is denoted by Leafj- and the set of inner nodes 
by Inner-r- If A = {e} we define Leafr = Innerj- — e, otherwise we define 
Innerj- := A \ Leafr. For a node a we define Children{a) — {ai\ai G A}. 

In the following we only consider L-labelled trees with finite branching, i.e. for 
all a G A : \Children(a)\ < oo. Next we will consider (S x M> x (Act x U {^l-- 
labelled trees T : A — > (S x M>0 x (Act x U{±})). We introduce abbreviations for 
the projections on the different components of the label (iii being the projection 
on the i-th component): Sta-f :— m o T, Probj- := ir2 °T and Act j- := tt^ o 7". 



Definition 9 (Transition Tree, cf. Definition 6 in [3,4j). Let M be a MA 

and VA{M) = (S, Act x ,->, 0, s ). A transition tree T over M is a (S x R>0 x 
(Act x U {J-})) labelled tree that satisfies the following conditions: 

1. Prob T {e) = I, 

2. Vct € Leafr : Act r (a) = 1 

3. Vcr G Innerj- \ Leaff : 3fx : Staj-(a) -5- ^ /i f— > being the transitions of 

VA{M)) and Prob T (cr) ■ (X = ® a > eChildren T (<T) Prob T(°'') • ^Sta T {<r') 

An internal transition tree over M is a transition tree where Vcr G Innerj- \ 
Leaf j- : Actj-(a) = r. Whenever the context is clear, we omit the term "over 
M". The distribution associated to a transition tree T is defined as fij- :— 
®a'eLeaf T Prob T (o- r ) ■ A staT ( a ,y We say that fj, r is induced by T . 

Now we can define weak transitions: 

Definition 10 (Weak Transition, cf. Definition 7 in [314] ). Let M be an 

MA, VA(M) = (5 1 , Act x , — 0, So) and s G S. We define weak transitions by 
looking at transition trees over M . We write: 

— s => /! if fx is induced by some internal transition tree T over M with 
Sta-r(e) = s. 

— s ix if ix is induced by some transition tree T over M with Stafie) = s, 
where on every path from the root to a leaf at least for one node a it holds 
that Actj-(a) — a. In case that a ^ r there must be exactly one such node 
on each of these paths. All other inner nodes must be labelled by r. 

— s =4> fx if (a — t and s =>■ fx) or (a ^ r and s =^>- fx) 

The difference between the a and a notation is only in the case a — r: A 
transition labelled with f can also remain in the same state without performing 
any transition, while label r requires at least one r transition to be performed. 
Next we define the notation of combined transitions: 

Definition 11 (Combined Transitions, cf. Definition 8 in |3,4|). Let M be 

an MA, VA{M) = (S, Act x , 0, s ) and s G S. We write s ^ c fx, if a £ Act x 
and there is a finite set I such that such that s (Xi for all i G / and ix is a 
convex combination of {fXi} ie i, i.e. 3{a G K + }i e / : J2iei Ci = 1 'A t = ®iei c tt L i- 
Combined s ==>c /i transitions are defined similarly. 

The above notations may be generalised from states to subdistributions: 

Definition 12 (Transitions between Subdistributions, cf. Definition 9 

in |3,4j). Let S be a set of states, fx G Subdist(S), ~^G {— >-, ==>, =>, =>c, =^c 
} . We write ix 7 if for all s$ G Supp([x) it holds that ~-+ % and 7 = 
®s l eSupp( P ,)t L {si)li. 



3 Relating nai've weak & weak bisimulation 



Remember that for an MA M its transitions have been defined by means of 
VA(M), so in the following it is safe to assume that all Markov Automata are 
represented by their PA images. All calculations will be made in this context. 
The first definition is a necessary adaptation of the definition in [3 4 that 

better corresponds to the definition of [5] . We use A instead of => because then 
it is allowed for the "defender" to remain in its state even if there is no explicit 
t loop (this corrected definition has also been given in [5]). 

Definition 13 (Naive weak bisimulation in the spirit of [8]). An equiv- 
alence relation 1Z on the set of states S of an MA M — [S,Act 1 — -►,-»-»-«-, sq) 
is called nai've weak bisimulation if and only if xlZy implies for all a £ Act x : 
(x A fi) implies (y =4>c //) with /i(C) = n'{C) for all C G s /u (note that the 
transitions are regarded in VA(M) ). Two MA are called naively weakly bisimi- 
lar if their initial states are related by a naive weak bisimulation relation on the 
direct sum of their states. 

We would like to note that modulo nai've weak bisimulation it is possible 
to omit r- loops in the image VA(-). The property whether a state is stable or 
unstable can still be recovered by looking for the presence (or absence) of x 
transitions. 

As naive weak bisimulation for MA is defined by means of the transformation 
VA(-), by comparing definitions the following lemma is clear. 

Lemma 1. Let MA be the set of Markov Automata. Naive weak MA bisimula- 
tion corresponds to weak probabilistic bisimulation on VA(MA) (as defined in 

w- 

Remark 3. This lemma does not hold for the original definition of nai've weak 
bisimulation in |3|4) . A minimal example is given in Fig. [2j According to the 
definition of |3|4j , Mi and M^ are not bisimilar (as M2 cannot perform a r tran- 
sition). In weak probabilistic bisimulation we have that VA{M{) and VA{M?) 
are bisimilar, as a r loop is implicitly assumed at every state. 

[314] argued that the (state-based) notion of naive weak bisimulation is too 
fine. Therefore they defined the coarser notion of (distribution-based) weak 
bisimulation: 

Definition 14 (Weak bisimulation |3j). A relation 1Z on sub- distributions 
over a set of states S of an MA M = (S, Act, — -»-, -»**-, sq) is called weak bisim- 
ulation if for all (jj,i,fi2) <E 1Z it holds that (transitions regarded in VA{M)) 

a.) \m\ = |ju 2 | 

B.) Vs e Supp(ni),Va e Act x : B^ifA '■ ^2 =^c M§ © lA suc h that 
(i) (fii(s) ■ A s )lZfir, and (/ii — s)TZ^i\ 

(ii) (s 4 fi'j) (3fi" : $ A c n" and (m(s) ■ fi[W) 



-0 



-0 



z 




x(0) 




-0 



(a) Mi 



(b) M 2 



(c) VA{M X ) 



(d) VA(M 2 ) 



Fig. 2: MA to PA transformations 



C.) a symmetric condition with fix and [12 interchanged ( roles of left-hand side 
and right-hand side also interchanged) 

Two distributions \x, 7 are called weakly bisimilar (with respect to some MA 
M), written fi w 7, if the pair (/1, 7) is contained in a weak bisimulation relation 
(with respect to M). Two states are called weakly bisimilar if their corresponding 
Dirac distributions are weakly bisimilar. We write s ~a t for A s « A t . Two 
MA are called weakly bisimilar if their initial states are weakly bisimilar in the 
direct sum of the MA . 

Remark 4- Theorem 1 in [314] shows that « is an equivalence relation (and 
therefore also So it is possible to talk about quotients. An equivalence 

class of a state s in s /k a is denoted by [s]~^. When the context is clear we will 
only write [s] without mentioning the quotient. 

A deep Theorem is Theorem 2 in |3I4) . The problem is that it is trivially 
wrong in the way it is written down there, as the following Remark shows. 

Remark 5. Look at the automaton given in Fig. [3] According to Def. we 
clearly have A w B, but Theorem 2 in [314] tells us that A 56 B as B ^> /i with 
/1 k, Ab- Moreover, the one-way condition stated in part 1 of Theorem 2 in |3I4) 
is not sufficient, because it does not imply that m/\ is an equivalence relation. 



To face the issues raised in Remark [5] we state Theorem [TJ 

Theorem 1 (Corrected version of Theorem 2 in [3,4]). Let M = (5, Act, 
, sq) be a MA and PA(M) = (S, Act x , — >, 0, sq) its corresponding PA. Let 
further be E, F £ S and fi, 7 G Dist(S). Then 




Fig. 3: Counterexample to Theorem 2 in [4] 



1. E F iff the following two implications hold: 

(a) whenever E A /i then for some 7 : F M>c 7 an d A* ~ 7 
f&j whenever F A 7 A/ien /or some /1 : P Ac A* an( ^ 7 ~ A* 

& ^ « 7 # V, V : A* A*'. 7 => 7', A* « /A 7 « 7' VC g S/*^ : M '(C) = 
7'(C) 

A direct consequence of the Theorem is the following corollary (already in- 
dicated in [4]) 

Corollary 1. If two MA Mi and M2 are naively weakly bisimilar, they are 
weakly bisimilar. 

Proof. Immediate by Def. 1131 and Theorem [TJ Simply take p! := \i and 7' := 7 

Remark 6. Combined transitions =>c can be used to rescale loops. A basic ex- 
ample is given in Fig. |4j An infinite transition tree rooted at s leads to the 
distribution hA x © \A y . To mimic the r transition of s, t has to perform a 
transition combined of | times t — ► \A X © \A y and | times A => Z\ t . Without 
using combined transitions A: could not mimick this transition of s. 




Fig. 4: Resolving r loop 



4 Vanishing states and their elimination 

We now introduce a notion of vanishing states in the context of MA. 

Definition 15 (local change of transitions). Let P = (S, Act, — >, 0, sq) be a 

PA, s g S , s unstable. We denote the transitions emanating from s by 

T s := {(s, a, A*) 1 3// g Dist(S),a g AcA : s A 

M^e define the set of weak combined transitions in P emanating from s as £(s) := 
{(s,t,v) g {s} x {r} x Dis^S) |s =^c f}. For any T s new C £(s) we dearie A/ie 
PA 

P(77H := (5, Act, (-> \T S ) U T? ew , 0, « W 

where we rename s to s' (assuming that s' is a new symbol). lfl~™ ew — {(s, r, ^)} 
we aZso wriAe P{ s ,v) — or simpliy P' if the context is clear — instead of P{{{s, r, v)}). 



Definition 16 (vanishing state). Let P = (S, Act, — >, 0, sq) be a PA. Let s € 

S be unstable and T s as in Def. \15\ State s is called 



trivially vanishing if T s = {(s, t, f)}. 

vanishing if there exists (s, r, v) € £(s) such that s ~a s' when comparing P 
and P( s ,v)- In this case P( s ,v) — or (s,v), for short — is called a vanishing 
representation of s. 

non-nai'vely vanishing if it is vanishing and there is a vanishing representa- 
tion P(s,v) with 3xq £ Supp(v): s xq. fnn- vanishing, for short) 

Example 2. Assume that p G (0,1). State E in Fig. [5a] is trivially vanishing 
since it only has an emanating r transition. A vanishing (but not nn-vanishing) 
state E is given in Fig. I5bl Note that all non-r transitions emanating from 
E may be omitted as they can be mimicked by appropriate weak transitions. 
The automaton in Fig. [5a] is the corresponding vanishing representation. E is 
naively vanishing as it turns out to be in the same class as C and D modulo 
weak bisimulation. For the last example, first note that in Fig. [Sc] C and D 
cannot be weakly bisimilar (because C can only perform the c to A, while D can 
additionally perform the d to B). As E is trivially vanishing we notice that it is 
also nn-vanishing, because E moves to the distribution pAc © (1 —p)A]j, where 
C 76 D. Moreover, since in Fig. [5c] D is not vanishing (and E is nn-vanishing), 
we have that E ^ D. 




(c) E nn-vanishing 
Fig. 5: Examples of vanishing states 



Example 3. Assume again that p € (0, 1). The example in Fig. [Sal shows that 
the property of being nn- vanishing is preserved by weak bisimilarity. This means 
that nn-vanishing states may be grouped into vanishing classes modulo weak 
bisimulation. We saw in the example before that E is an nn-vanishing state. 
Obviously E and F belong to the same class modulo weak bisimulation. We 
want to know what the vanishing representation for F is. Now we ignore E, 
cf. Fig. I6bl (Definition [T71 and Lemma [3] will justify that this is reasonable), and 
rescale the r transitions emanating from D according to Remark [5] (cf. Fig. I6"c)) . 
We see that there is a r connection from F to states C and D and it holds that 
A F K,p-A c ®{l-p)-A D . So we use T s new = {(F,r,p- A c 8 (1 -p) • A D )} to 
get the vanishing representation for F. Note also that in Fig. [6a] we need two r 
steps to get the weak transition that can be used as vanishing representation of 
F. This indicates that it is not trivial to find vanishing representations. 




(b) E ignored (c) D rescaled 



Fig. 6: Examples of nn-vanishing states 



Remark 7. As naive weak bisimulation is defined on states rather than distribu- 
tions it is easy to see that a nn-vanishing state s cannot be vanishing in the naive 
sense: only if all successor states still lay in the class of s it would be possible, 
as the distribution then could be identified as one single state. 

The use of combined transitions £(s) for vanishing representations is too 
general, as the following lemma shows: 



Lemma 2. Every vanishing representation (s,/i) can be transformed to a (non- 
combined) vanishing representation (s, 7') where s =>• 7' is induced by a Dirac 
determinate scheduler in the sense of £fj/. 

Proof. Look at the diagram in Fig. [7] Firstly we show that we can restrict 
ourselves to the non-combined case. Let P( s ,n) be the vanishing representation 
of s (so, by definition, A s « fi). If the corresponding transition is not combined, 
we are done. So we may assume that the vanishing representation is given by a 
combined transition (indicated by the C in the upper arrow). The down arrows 
are granted by part 2 of Theorem [T] (also A s rs 7' and fi rs //). The identity in 



I id 1 

7 fj, 

Fig. 7: Finding non-combined vanishing representations 



the lower line is on classes C £ s /~a (as described in part 2 of Theorem [TJ. By 
transitivity of ~ it is clear that fi' is also a vanishing representation. But then 
we also may take 7' as the vanishing representation, which is reached by a single 
transition tree. This construction also works with Dirac determinate schedulers 
(observe that in the proof of Lemma 16 in [3] Dirac determinate schedulers have 
been used to construct the transition trees), so the lemma is shown. 

Corollary 2. Let P = (S, Act, — ¥, 0, sq) be a PA. State s G S is nn-vanishing 
if and only if there is a vanishing representation of s generated by a Dirac de- 
terminate scheduler that leaves the equivalence class of s in s /^a with positive 
probability. 

Proof. Just a reformulation of Definition [16] using Lemma [2j 

Definition 17 (Elimination of vanishing states). Let P = (S, Act, — >, 0, sq) 

T 

be a PA. Let s s S be a vanishing state and let s — >' v be the only transition 
emanating from s in the vanishing representation P' = P( s ,v) = {S, Act, — >' 
,0, so). The elimination of s is defined by two steps: 

1. Reseating (cf. Remark\Bj): 



' res 



->'\{(s,r,v)} ifv = A s 

[-+' U {{s,t, 1 _l {s) (v - s))} otherwise 



2. Elimination ( only performed if after rescaling a transition s — ¥ res v re- 
mains): 



(S\{s},Act,^",<b,s ) ifs^s 
((5\{s })U {s° Q },Acb,^" U{(s° ,t,v)}As o) ifs=s and 

P' otherwise 

where — >":= {(t, a, fi')\t —> res fi,t € S\{s} } /i' := /i s ^ u }. Here fi s -n, denotes the 
replacement of every occurrence of s by the corresponding distribution v: Without 
loss of generality let jj, be of the form jj, := c s A s © (®i£i, Si jtsCiA Si ) and v be of 
the form v = ®j e j, Sj7 i s djA Sj . Then we define pb s ^, v := c s (® j&JtSj jt s djA Sj ) © 

Remark 8. We omit the (s,t,A s ) transition from the set of transitions for the 
purpose of minimality of the resulting PA. One could also just add the loop case 
to the case where P' is not changed. Note that even when loops are removed, all 
information about the MA may be safely recovered. Such a state without loop is 
a deadlock in PA and no longer vanishing according to our definition (note that 
there cannot be any other competing transition, as we then could not get the 
vanishing representation with the r loop) . Looking back to the MA setting it is 
clear that s must be an unstable state as it does not have the x(0) transition. 

Example 4- To explain Definition [T7] we give the following examples. The first 
example is the most common one (cf. Fig. 18a)) : The vanishing state s is neither 
the initial state nor does it have a probability-one-self-loop. Therefore the elimi- 
nation is straight-forward: Redirect all incoming arcs according to the vanishing 
representation (cf. Fig. I8b[) . The next example is the probability-one-self- loop 
case (cf. Fig. l8a| : It does not matter whether the vanishing state s is the ini- 
tial state or not, the self-loop is removed by the rescaling operation (cf. I8d[) . In 
the third example we have a vanishing initial state with incoming transition(s) 
(cf. Fig. Eil). We add a copy Sq of the initial state and eliminate the old initial 
state so (cf- Fig. |8fJ . Note also that when s is a vanishing initial state but it has 
no incoming transitions, then nothing is changed (without a figure). 

Lemma 3 (Elimination does not destroy weak bisimilarity). For every 
vanishing state s it holds that P w P' s 

Proof. By definition P w P' . With the same arguments as in [3] (proof of The- 
orem 7) it follows that P' « P ls . So by transitivity of w the claim follows. 

Remark 9. For a given PA P the set of vanishing states is a superset of the set 
of trivially vanishing states. By definition, every non-trivially vanishing state 
s must have a vanishing representation where s is trivially vanishing. So, by 
successively replacing the P by vanishing representations, only trivially vanishing 
states remain. 




1 




(a) Case 1 (b) Case 1 eliminated 



(c) Case 2 (d) Case 2 eliminated 




(e) Case 3 (f) Case 3 eliminated 



Fig. 8: Different cases of eliminations 

The machinery up to now motivates the following definition: 

Definition 18. Let P = (S, Act, 0, s ). Let S v = {s\,...,s%} be the set 
of vanishing states. Denote by P the complete elimination of P, i.e. P := 
(. . . (P' s \y s 2 . . Let P* denote the elimination of all nn-vanishing states. 

From this we get the following relation: 

Lemma 4 (Complete eliminations and bisimilarity). For two PA Pi and 

P 2 it holds: P x « P 2 ^> Pi « P 2 ^ P* « P* 

Proof. By Lemma [3] we know that elimination preserves weak bisimilarity. The 
following diagrams show this by the right arrows. 

Pi Pi* Pi 



Pi — Pi P-2 

As soon as one of the down arrows is a weak bisimulation, by transitivity of 
weak bisimulation we immediately get that the other two arrows are also weak 
bisimulations. 

Remark 10. In every example from [3], elimination leads to isomorphic automata 
(assuming that we replace vanishing initial states by their vanishing representa- 
tion). 



It is clear by Corollary [T] that Pi « n aiVe Pi =>■ Pi ~ P2 an d therefore, by 
Lemma H] Pi « P2. Now we try to understand why it is also the case that 

P\ ~ P2 Pi ~ naive Pi- 

Lemma 5. It is in general not possible to eliminate all unstable states. 

Proof. The simplest example possible is the one given in Fig. [5] Assume there 
was a weak bisimulation relation 1Z = {(A s ,A t ), . . .}, where (Ae, A f ) £ 1Z and 
(Af,Ae) ^ 7Z- From the definition of weak bisimilarity we would deduce that 
also (A Sl , cAe © (1 — c)Af) € TZ for some c £ [0, 1]. But now observe that we 
would need to have both (A E ,cA E © (1 - c)A F ) and (A Fl cA E (1 - c)A F ), 
therefore c = and c = 1 , which is a contradiction. 




Lemma 6. Only vanishing states can be eliminated 

Proof. Let P be the MA of interest. Suppose that a state s is not vanishing, 
then for all (s, r, v) € (£(s) we have P 76 P(s, r, z/). Using the notation of Dcf. [16] 
and taking T s new C C(s) (with 17;"^ | > 2) and assume that p(T? ew ) ~ P, 
but then - as there does not exist a vanishing representation - in s at least two 
nondeterministic choices must be possible that lead to different behaviour. So 
with the same argument as in the proof of Lemma [5] we conclude that s cannot 
be eliminated. 

Theorem 2. It holds that Pi « P 2 <^> P* fa n <ave Pi- 
Proof. <= is immediate by Corollary [T] and Lemma |4j 

=$> From Lemma [4] we already know Pi w P2 P* ~ P2*- So it remains to show 
that Pj* « P 2 * is already a naive weak bisimulation. By part 1 of Theorem Q] we 

must have for a pair of states s ~a t that for every s A /1 we find t 7 with 
w 7 (and vice versa). Assume that we split /i according to its support: /1 = 
©i g /CjZ\ Si , then with Lemma 11 of [4] we get a transition 7 =$-c l' = ffiie/ c i7i 
with Z\ Sj w 7 Z '. By part 2 of Theorem [T] we get the following diagram for every 
i el: 



The identity in the lower line is on classes C G s /~a- By assumption Sj cannot 
be nn-vanishing (only the initial state could be, but Si ^ sq by definition of the 
elimination), so there cannot be a vanishing representation leaving the equiva- 
lence class of Si, in other words for every i € /: //-(C) = A Si (C) — 7"(C) for all 

C e s /~a- Defining 7" = ©ie/c^" we get t 4> 7 => c 7' =>■ 7", that is t 7" 
and VC £ s/^ : 11(C) =Y'(C). 

We can use the same argumentation for i 4 7 to find s =l>c m" with 
VC G s /~zi : 7(C) = //"(C) an d we conclude that is already a naive weak 
bisimulation relation. 

Corollary 3. It holds that Pi ss P2 •O- Pi Wnowe P2 ■ 
Proof. <= is immediate by Corollary [T] and Lemma 01 

=>■ The hard part is Theorem [2] as non- naively vanishing states may be ignored 
and therefore some equivalence classes modulo weak bisimulation may be ig- 
nored. As naively vanishing states by definition cannot change their equivalence 
class modulo weak bisimulation, eliminiation will not change anything - the 
equivalence class remains and cannot be ignored. 

5 A partition refinement algorithm 

With Corollary [5] we can find tangible states and Theorem [5] reduces the problem 
to naive weak bisimulation. Even if the problem of deciding naive weak bisimu- 
lation as recently been shown to be solvable in polynomial time [S] , the decision 
problem for weak bisimulation remains exponential as finding vanishing rep- 
resentations involves all dirac determinate schedulers (which are exponentially 
many, as has been shown in pQ). For the description of the partition refinement 
algorithm below we need the convex sets S(x, a) C M. n introduced by pQ: 

Example 5. For the MA given in Fig. |5] (left-hand side) the resulting set S(si, r) 
is given by the shaded triangle in Fig. [TUJ This triangle encodes all distributions 
that are reachable via a (weak) combined r transition starting from s\, for details 
on how to calculate those sets we refer to pQ. 




Fig. 10: S( Si ,t) 



Remark 11. In [1 it is shown that the convex sets are the convex hull (CHull) 
of distributions (i.e. points in K n ) generated by Dirac determinate schedulers. 
It is shown there that the extremal points (i.e. generators) of the convex hull 
can be found by a linear program and that the complexity of calculating the 
sets S(s, a) is exponential for the weak case. Therefore our algorithm also has 
exponential complexity. 

Remark 12 (Restriction of convex sets). The sets S(s, a) C 1" may be re- 
stricted. We define 

S(s,a)\ Xi=0 := {(x%, ...,£„) <E S(s,a)\xi = 0} 

Of course, multiple restrictions can also be realised. Especially, we define a re- 
striction to a set of "tangible" states S(s, ot)\s ta „ gible by requiring that Xi — for 
all nn- vanishing states xi . Note that the restriction of convex sets is still convex 
by definition. 

Our algorithm works according to the partition refinement principle - looking 
at restrictions of convex sets - as follows: 

1. Start with the initial partition W = S\ U 52. 

2. For all states s and actions a calculate the convex sets S(s,a) (cf. [T]) and 
and for every (Dirac determinate) weak transition (s, r, v) calculate S u (s, a) 
(S v (s,a) denotes the convex set calculated for the PA (Pi U P2){ s ,v)i that 
is the direct sum of automata P\ and P2 where we move to the vanishing 
representation (s,u)). 

3. Set Stangible = 0- 

4. For all states s that are not in Stangible 

— Check whether s can leave its equivalence class in W with some prob- 
ability > by a (Dirac determinate) weak transition (s,t,v) such that 
(modulo W) S(s,a)\s tangible = S u (s,a)\s tangible for all a 6 Act. This is 
a vanishing representation with respect to W. 

— If no vanishing representation with respect to the current partition W 
can be found, s must be tangible with respect to W. Add those states 

to Stangible- 

— Cross-check all other states if they also become tangible as an effect of 
s being tangible. 

5. Find a new splitter (in the sense of [1]) with respect to the current partition 
and the current set of tangible states, i.e. a tuple (C, a, W) which indicates 
that class C needs to be refined w.r.t. a weak a transition. 

6. Refine the partition according to the splitter and start next round. 

The algorithm can be found in Alg. [T] By DiracDet{s, r) we mean all dis- 
tributions v induced by s => v by means of a Dirac determinate scheduler. It 
remains to define the Computelnfo algorithm, FindWeakSplit algorithm and the 
Refine algorithm. As the Refine algorithm is standard, we omit it from this pa- 
per. The routine Computelnfo just calculates the convex sets S(s, a) according 
to Remark [TT1 The routine FindWeakSplit given in Alg. [5] pretty much looks like 
the one given in [T] but it only "sees" tangible states that are provided as an 
additional parameter to the routine. 



Algorithm 1 DccideWeakBisim 



Require: Two MA as PA Pi = (Si, Acti, -H, 0, s ), P2 = (5 2 , Aci 2 , -> 2 , 0, *o) 



1: <S = Si U <S 2 , W = 5, Act = AcU U Act 2 

2: for s £ S, a G Act, 1/ G DiracDet(s,r) do 

3: /S(s,a) = ComputeInfo(s,a) 

4: £„(s,a) = ComputeInfo(s,a) on (Pi U ft)),,^ 

5: end for 

6: while W changes do 

7: Stangible ■— 

8: while Stangible changes do 
9: for s G <S \ Stangible do 

10: for i/ G DiracDet(s,r) where 3x G Supp(v) : [x]w 7^ [s]w do 

11: if Va G Act : (S( a , a) | Sta „ gi6! J/W = (&,(«, a)|s tongi6ie )/W then 

12: vanishing representation found break 

13: end if 

14: end for 

15: if no vanishing representation found then 

16: 

^tangible • ^tangible 

u{s} 

17: end if 

18: end for 
19: end while 

20: (C, a, W) = FindWeakSplit(S tangMe , W, S, Act, S(; ■)) 

21: W = Refine(C, a, W) 

22: end while 

23: Pi « P 2 iff [s ]w = [to]w 



Algorithm 2 FindWeakSplit (Find weak bisimulation splitter) 

Require: Tangible states Stangible, partition W, states S, actions Act, Info S(-, •) 
1: for d G W, s,t £ Cj, a G Act do 

2: if (5( S ,a)| StaBe4Wa )/W 9 4(5(t,a)| 5taBe4Wa )/Wthen 
3: return (d, a, W) 
4: end if 
5: end for 



Remark 13. The algorithm detects nn- vanishing states and finds their vanishing 
representations (regarding the last partition W). Therefore we will be able to 
eliminate all nn- vanishing states and reach a form where only tangible states are 
present and where for every equivalence class only one state is used. This can be 
regarded as a kind of normal form. 

5.1 Example 

Suppose we are given the MA in Fig. Illal (there already transformed to a PA P) 
where p,q e (0, 1) (ignore for the moment the two initial states). This automaton 
can be seen as a condensed form of two different automata (starting with s% and 
ti), where states A and B have been identified (to keep things short - if there 
were two copies of A and B: one for the left and one for the right automaton, 
they would be grouped during the progress of the algorithm). We want to show 
that si ~a t\. 




(a) Non-trivial example 




(b) Wo (c) Wi (d) W 2 



Fig. 11: Example and partitions during algorithm run 

We assume that p = q = h, as the pictures are easier to draw in that case, 
but we would like to stress that the same arguments work for all other choices 
(as long as p and q are not equal to or 1). 

Remark 14- For didactical reasons we add dots for every result of a Dirac de- 
terminate scheduler (according to [T]) whenever we draw convex sets. Dots that 
are not extremal points may safely be omitted, as we are talking about convex 
sets. 



First round: Start with the partition Wo = {{si, s 2 , ti,A, B}} (cf. Fig. lllbl) . 
Observe that in the loop from line 9 to line 18 we can never find a vanishing 
representation, as no state may leave its equivalence class with some probability 
greater than zero. Therefore we get StangiMe — { s ii s 2 ,ti, A, B}. 

Now we have to find a splitter with respect to {StangiMe, Wo). Suppose that 
we check the sets S(-,b)/Wo- Here we identify si = s 2 = t\ = A = B and see 
that: 




S(x, 6) /Wo = <j V!2V f or x€ {B,si,s 2 ,ti} 

otherwise 

So we have found a splitter. Refining according to ({sx, s 2 , *i, A, B}, b, Wo) leads 
to Wi = {{ Sl ,s 2 ,ti,B},{A}} (cf. Fig.[TTi. 

Second round: We first have to detect the nn- vanishing states with respect 
to the current partition. We calculate S(x, r) for every state, verify if it is possible 
to reach another equivalence class and see whether one single r transition suffices. 
The values of S(x, r) are given in Tab.[TJ Firstly notice that both A and B cannot 



S(x,t)/Wi 



Sl 




(J, J) 



A 



S{x,t)/Wi 



r® 



r® 



Table 1: S{x,r) 



A'2 



S(x,r)/m 



be nn-vanishing, as they have no possibility of leaving their equivalence classes. 
Notice also, that even if s 2 is vanishing, as it has only one single emanating r 
transition, we cannot detect it as nn-vanishing (the only van. rep. that leaves 
the class would be P {s2 ^ Aa(B * Ab) , but S(s 2 ,b)/Wi ^ Si AA(B 2 AB (s 2 ,b)/Wi). 
Regarding si we see that we cannot omit transition s\ A ^Aa® \A S2 , a s 
S(a 32 )(si,t)/Wi = S(B,t)/Wi ^ S(si,t). But notice also that si A A S2 
cannot be omitted, as S(si, b)/W\ = S(B, r)/Wi, but 5(i AaS) i/j a )(si, b)/Wi = 
0. So we see that si cannot be nn-vanishing. With the same argument we see that 
also ti cannot be nn-vanishing. Therefore we get StangMe = {si, S2, ii, A, B}. 

Now we look for splitters with respect to (Wi , StangiMe) ■ Looking at C = 
{si, S2, h,B} we see in routine FindWeakSplit that we can use a splitter (C, r, Wi) 
and get the partition W 2 = {{s 1 ,t 1 },{s 2 },{A},{B}} (cf. Fig. Illdl note that 
S(si,t)/Wi = S(ti,r)/Wi, as (|, ^) is not a generator of the convex set). 



Third round: We first have to detect nn-vanishing states. It is clear that S2 
must be nn-vanishing as it can leave its class and only has a single outgoing r 
transition. With the same arguments as above we see that both s± and t\ must 
be tangible. So we get Stangibie = {si, h,A, B}. 

Now we again can look for splitters, but have to consider the restriction to 
Stangibie - Notice that with coordinates [s\] — [t\], [S2], [A], [B] we have 

f X \ (°\ (°\ (°\ 
1 

' ' 

\V W W 



S{s u t)/W 2 = CHull{ 



f 
2 

W 





2 



We want to calculate the restriction S(si,T)\s tangible /W2- Let us for the mo- 
ment ignore the vertex [si] G S(s-l,t)/W2- Then we get the picture in Fig. I12al 
for 5(si,r)| Sl =o/W2- We see that the restriction of this set to Stangibie gives 




(a) 5(si,r)| sl=0 /W2 




(b) S( Sl ,T)\ sl , a2=0 /m 
Fig. 12: Convex sets 




® 

(c) s( Sl ,T)\ Stangibl jm 



only the line from (0,0,1) to (0, |, |) (cf. Fig. IT2bl) . therefore we conclude 
that S(si,T)\s tan 4Me /Wa is the set given in Fig. I12cl We get the same set for 
-S^ii r )l<s t „„ g i6ie/^ ; 2 (here, no nn-vanishing state has to be ignored). Looking at 
all other sets S(-, •) we find no other splitter, so W2 cannot be refined. 

With the partition W2 and the set of stable states S tan gibie we have reached 
our fixed point, the algorithm terminates and we see that s\ and t\ are still in 
the same partition, so they are weakly bisimilar. 

Remark 15 (Optimisations) . A few optimisations can be performed: 

— In every round states without other r transitions than the loop will always 
be detected as stable, so this set can be separated as a preprocessing step. 

— All states with only one single outgoing r transitions and without non-r 
transitions can safely be eliminated in advance (cf. Lemma 13]). 

6 Conclusion 

We have shown that weak and naive weak bisimulation for MA are closely re- 
lated by an appropriate formulation of elimination and that the two notations 



coincide, when no non- naively vanishing states are present. We have presented 
an algorithm for deciding weak MA bisimilarity that, as a by-product, hnds 
non-nai'vely vanishing states and their corresponding vanishing representations. 
This can also be used to define normal forms for MA. Even with the magnificent 
results of [5] it remains an open question whether weak MA bisimulation can be 
decided in polynomial time. 
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A Timed actions with rate zero 

The following remark shows why we modified the definition of P s (see Def. |H 
for the original definiton see [213 4 ). To explain the argument, we need to recall 
the definition of parallel composition of MA (cf. Definition 2 in [3]). As in our 
example we only use the unsynchronised case, we only note that a state in the 
composed MA may perform all transitions of its subcomponents. As usual, we 
denote parallel composition by the || operator. 




VA{Mx\\M A ) VA(M 2 \\M 4 ) 
Fig. 13: Counterexample to Theorem 3 in [314] (unless %(0) is defined) 



Remark 16 (Parallel composition and x(0)J- We discuss briefly the case when a 
stable state has no outgoing timed transition. Note that the case rate(u) = 
remains undefined in 2 3 4 (division by zero), therefore no x(0) action can be 
generated there. This would be problematic, as without the x(0) action we would 
have VA(Mi) m VA(Mq) in Fig. [TJ but parallel composition with the M4 from 
Fig. I13al would lead to different results: Afi 1 1 A/4 is given in Fig. I13bl M2WM4 
is given in Fig. I13cl The corresponding PA are given in Fig. I13dl and Fig. I13cl 



respectively. Note that the results are different modulo weak bisimulation and 
thus falsifying Theorem 3 in |3I4) which claims that weak bisimulation is a con- 
gruence with respect to parallel composition. So we would like to stress that it is 
necessary to use Def. @]for the successor probability and not the definitions from 
[2 3 4]. Only with this definition Theorem 3 in |3I4) can hold. We conjecture that 
the results of |3I4I2] still hold using Def. [U but we did not prove that. We only 
have shown by the counterexample that without considering x(0), the results 
cannot hold. 

References 

1. S. Cattani and R. Segala. Decision algorithms for probabilistic bisimulation. In 
L. Brim, P. Jancar, M. Kretmsky, and A. Kucera, editors, CONCUR, volume 2421 
of Lecture Notes in Computer Science, pages 371-385. Springer, 2002. 

2. C. Eisentraut, H. Hermanns, and L. Zhang. Concurrency and composition in a 
stochastic world. In P. Gastin and F. Laroussinie, editors, CONCUR 2010 - Con- 
currency Theory, volume 6269 of Lecture Notes in Computer Science, pages 21-39. 
Springer Berlin / Heidelberg, 2010. 

3. C. Eisentraut, H. Hermanns, and L. Zhang. On probabilistic automata in contin- 
uous time. In Proceedings of the 2010 25th Annual IEEE Symposium on Logic in 
Computer Science, LICS '10, pages 342-351, Washington, DC, USA, 2010. IEEE 
Computer Society. 

4. C. Eisentraut, H. Hermanns, and L. Zhang. 
On Probabilistic Automata in Continuous Time. 

http: //www. avacs . org/f ileadmin/Publikationen/0pen/avacs_technical_report_062 .pdf , 
2010. Reports of SFB/TR 14 AVACS 62. 

5. H. Hermanns and A. Turrini. Deciding Probabilistic Automata Weak Bisimulation 
in Polynomial Time. |http : //arxiv . org/abs/1205 . 0376 2012. arXiv: 1205.0376. 

6. J. Schuster and M. Siegle. Markov Automata: Deciding Weak Bisimulation by 
means of "non-na'ively" Vanishing States, http://arxiv.org/abs/1205.6192vl, 
2012. rejected CONCUR'12 submission. 

7. R. Segala. Modeling and Verification of Randomized Distributed Real-Time Sys- 
tems. PhD thesis, Department of Electrical Engineering and Computer Science, 
Massachusetts Institute of Technology, 1995. 

8. R. Segala and N. A. Lynch. Probabilistic simulations for probabilistic processes. 
Nord. J. Comput., 2(2):250-273, 1995. 



